Home | | Latest | About | Random
# 25 Left matrix multiplication maps.
## Characterization of linear maps between Euclidean spaces.
We have this characterization of all linear maps $T:\mathbb{R}^{k}\to \mathbb{R}^{n}$ are just left matrix multiplication maps:
> **Characterization.**
> A function $T:\mathbb{R}^{k}\to \mathbb{R}^{n}$ is a linear transformation if and only if there exists a matrix $A$ of size $n\times k$ such that for all $\vec x \in \mathbb{R}^{k}$ we have $T(\vec x) = A\vec x$.
In particular we will see that this matrix $A$ has columns that are given by $$
A = [\ T(\vec e_{1})\ T(\vec e_{2})\ \cdots\ \ T(\vec e_{k})\ ]
$$where $\vec e_{i}$ are the $i$-th columns of the $k\times k$ identity matrix. We will call this matrix $A$ the **standard matrix of $T$**. We will also write $[T]_{\text{std}}$ to represent the standard matrix of $T$.
Let us prove this.
$\blacktriangleright$ Proof.
$(\implies)$ Suppose $T:\mathbb{R}^{k}\to \mathbb{R}^{n}$ is a linear transformation. We will construct a suitable matrix $A$ that $T(\vec x) = A \vec x$ for all $\vec x \in \mathbb{R}^{k}$.
Denote $\vec e_{i} \in \mathbb{R}^{k}$ to be a column vector where the $i$-th entry is $1$ and the rest is zero, in other words, $\vec e_{i}$ is the $i$-th column of the $k\times k$ identity matrix $I_{k\times k}$. So $$\vec e_{1}=\begin{pmatrix}1\\0\\0\\\vdots\\0\end{pmatrix},\vec e_{2}=\begin{pmatrix}0\\1\\0\\\vdots\\0\end{pmatrix} ,\ldots \text{etc.}$$These are sometimes called the **standard basis vectors** in $\mathbb{R}^{k}$.
Now consider the column vectors $T(\vec e_{i})$ for each $i=1,2,\ldots,k$. Note each $T(\vec e_{i}) \in \mathbb{R}^{n}$. Now construct the matrix $n\times k$ matrix $A$ to have these $T(\vec e_{i})$ as columns: $$
A = [\ T(\vec e_{1})\ T(\vec e_{2})\ \cdots\ \ T(\vec e_{k})\ ]
$$
We claim this matrix $A$ does the job. Indeed, for any $\vec x \in \mathbb{R}^{k}$, say $$
\vec x = \begin{pmatrix}x_{1} \\ x_{2}\\\vdots \\x_{k}\end{pmatrix}
$$note that $\vec x = x_{1}\vec e_{1}+x_{2}\vec e_{2}+\cdots + x_{k}\vec e_{k}$. So if we apply $T$ to $\vec x$, we get $$
\begin{align*}
T(\vec x) &=T(x_{1}\vec e_{1}+x_2 \vec e_{2}+\cdots x_{k}\vec e_{k}) \\
&= x_{1}T(\vec e_{1}) + x_{2}T(\vec e_{2}) + \cdots x_{k}T(\vec e_{k}) \\
&= [\ T(\vec e_{1})\ T(\vec e_{2})\ \cdots\ \ T(\vec e_{k})\ ] \begin{pmatrix}x_{1}\\x_{2}\\\vdots\\x_{k}\end{pmatrix} \\
\implies T(\vec x) & =A\vec x.
\end{align*}
$$Hence for any linear $T:\mathbb{R}^{k}\to \mathbb{R}^{n}$ there is such a matrix $A$ such that $T(\vec x)=A \vec x$.
$(\impliedby)$ Consider the function $T:\mathbb{R}^{k}\to \mathbb{R}^{n}$ with $T(\vec x)=A\vec x$ for some $n\times k$ matrix $A$. We will show $T$ is a linear transformation. Indeed, for any two vectors $\vec x$ and $\vec y$ from $\mathbb{R}^{k}$, we have $$
T(\vec x + \vec y)=A(\vec x + \vec y)= A\vec x+ A\vec y=T(\vec x) + T(\vec y).
$$So $T$ preserves sum. And if we consider any vector $\vec x \in \mathbb{R}^{k}$ and any scalar $c$, then $$
T(c\vec x)=A(c\vec x)=cA\vec x=cT(\vec x).
$$
So $T$ preserves scaling. Hence $T$ is a linear transformation. $\blacksquare$
**Example.**
Suppose we have a linear map $T:\mathbb{R}^{3}\to \mathbb{R}^2$ given by $$
T\begin{pmatrix}x\\y\\z\end{pmatrix} = \begin{pmatrix}x+4y+2z\\3x-2y+z\end{pmatrix}
$$Find a matrix $A$ such that $T(\vec x)=A\vec x$. That is, find the standard matrix of $T$.
$\blacktriangleright$ Since we assumed $T$ is linear (and one can check that it is), then using the result of the characterization above we can directly write down its standard matrix $A$, given by $$
A=[T(\vec e_{1})\ T(\vec e_{2})\ T(\vec e_{3})].
$$Calculating the columns give $T(\vec e_{1})=\begin{pmatrix}1\\3\end{pmatrix}$, $T(\vec e_{2})=\begin{pmatrix}4\\-2\end{pmatrix}$, and $T(\vec e_{3})=\begin{pmatrix}2\\1\end{pmatrix}$, so $$
A=\begin{pmatrix}1 & 4 & 2\\3 & -2 & 1\end{pmatrix},
$$and $T(\vec x)=A\vec x$ with $A$ as above. $\blacklozenge$
**Remark.** This is column-wise computation of the standard matrix of a linear map between Euclidean spaces is especially useful when the rules of the map is described geometrically.
## Kernel and range of a linear map.
Let $T:U\to V$ be a linear map between linear spaces $U$ and $V$ over the same set of scalars. We define the **kernel of $T$** to be the set of inputs in $U$ that get sent to $\mathbf{0}_{V}$, and we denote $$
\ker T = \{u\in U : Tu = \mathbf{0}_{V}\}.
$$And we define the **range of $T$** to be the set of actual outputs by the map $T$, as usual, and we denote $$
\operatorname{ran} T = \{Tu:u\in U\}.
$$
Note for a linear map $T:U\to V$ whose domain is $U$ and codomain is $V$, the kernel of $T$ is a subset of $U$, while the range of $T$ is a subset of $V$. In fact, they are both linear spaces. And since $\ker T$ is a linear space inside the linear space $U$, and $\operatorname{ran} T$ is a linear space inside the linear space $V$, we say they are **subspaces** of their respective larger container linear space.
> **Proposition.**
> Given a linear map $T:U\to V$, both $\ker T$ and $\operatorname{ran} T$ are linear spaces.
$\blacktriangleright$ Proof. We show they (1) contain the zero element, (2) are closed under addition, and (3) are closed under scaling.
Let us look at $\ker T$ first.
(1) Note $\mathbf{0}_{U}\in \ker T$ since $T(\mathbf{0}_{U})=\mathbf{0}_{V}$.
(2) Take $x,y\in \ker T$, then $T(x)=\mathbf{0}_{V}$ and $T(y)=\mathbf{0}_{V}$. Then their sum $x+y$ is such that $T(x+y)=T(x)+T(y)=\mathbf{0}_{V}+\mathbf{0}_{V}=\mathbf{0}_{V}$, by using linearity. Hence $x+y \in \ker T$.
(3) Take $x \in \ker T$ and any scalar $c$. Then $T(cx) =cT(x)=c \mathbf{0}_{V}= \mathbf{0}_{V}$. Hence $cx \in \ker T$.
So $\ker T$ is a linear space.
Let us look at $\operatorname{ran} T$.
(1) Since $T(\mathbf{0}_{U})=\mathbf{0}_{V}$, so we see that $\mathbf{0}_{V}$ is the output of some input, and so $\mathbf{0}_{V}$ is in $\operatorname{ran} T$.
(2) Take $v,w \in \operatorname{ran}T$. Then each of them is the result of some inputs, say $x,y \in U$ such that $T(x)=v$ and $T(w)=y$. Then note that $T(x+y)=T(x)+T(y)=v+w$, so their sum $v+w$ is the result of the input $x+y$, hence $v+w \in \operatorname{ran}T$.
(3) Take $v \in \operatorname{ran}T$ and $c$ any scalar. Then again we have some $x \in U$ such that $T(x)=v$. Note that $T(cx)=cT(x)=cv$, so $cv \in \operatorname{ran} T$.
So $\operatorname{ran} T$ is a linear space. $\blacksquare$
Since $\ker T$ and $\operatorname{ran} T$ are both linear spaces, they can be written as a span of some spanning set, and further we can find a basis for these linear spaces and discuss their dimensions. We define the dimension of $\ker T$ as the **nullity of $T$** and the dimension of $\operatorname{ran}T$ as the **rank of $T$**.
## Kernel and range of Euclidean linear maps.
Suppose further that we have linear map $T:\mathbb{R}^{k}\to \mathbb{R}^{n}$ between Euclidean spaces. Then by the characterization earlier we have some $n\times k$ matrix $A$ where $T(\vec x) = A\vec x$ for all $\vec x \in \mathbb{R}^{k}$. Here we can concretely answer what the kernel and range of the linear map $T$ is:
> **Proposition.** For a linear map $T:\mathbb{R}^{k}\to \mathbb{R}^{n}$ given by $T(\vec x) = A\vec x$ for some $n\times k$ matrix $A$, we have $\ker T = \text{nullspace}(A)$ and $\operatorname{ran}T = \text{columnspace}(A)$.
$\blacktriangleright$ This is essentially the definition of the words $\ker T$, $\text{nullspace(A)}$, $\operatorname{ran}T$, and $\text{columnspace}(A)$. Note $$
\begin{align*}
\vec x \in \ker T & \iff T(\vec x) = \mathbf{0} \\
& \iff A\vec x = \mathbf{0} \\
& \iff \vec x \in \text{nullspace}(A)
\end{align*}
$$And $$
\begin{align*}
\vec y \in \operatorname{ran} T & \iff \exists x\in \mathbb{R}^{k} \text{ such that } T(\vec x) = \vec y \\
& \iff \exists x\in \mathbb{R}^{k} \text{ such that } A\vec x = \vec y \\
& \iff \text{the equation } A\vec x = \vec y \text{ is consistent} \\
& \iff \vec y \text{ is a linear combination of columns of }A \\
& \iff \vec y \in \text{columnspace}(A)
\end{align*}
$$We have as claimed. $\blacksquare$
**Example.**
Consider the linear map $T:\mathbb{R}^{3}\to \mathbb{R}^{2}$ given by $$
T \begin{pmatrix}x \\
y \\
z\end{pmatrix} = \begin{pmatrix}x+y \\
y+z\end{pmatrix}
$$Find a basis of $\ker T$ and $\operatorname{ran} T$, and determine $\text{nullity}(T)$ and $\text{rank}(T)$.
$\blacktriangleright$ Since this is a linear map between Euclidean spaces, let us construct the matrix of $T$ first. By the characterization, the matrix of $T$ has columns given by $T(\vec e_{1}),T(\vec e_{2}), T(\vec e_{3})$. So we have $$
A = \begin{pmatrix}1 & 1 & 0 \\
0 & 1 & 1\end{pmatrix}
$$where $T(\vec x) = A\vec x$.
In this case $\ker T = \text{nullspace}(A)$. So we compute nullspace of $A$ by row reduction, and as it is already in echelon form, we have $z$ free, $y=-z$, $x=-y=z$. So $$
\text{nullspace}(A)= \operatorname{span} \{\begin{pmatrix}1\\-1\\1\end{pmatrix}\}
$$Hence a basis for $\ker T$ is $\{\begin{pmatrix}1\\-1\\1\end{pmatrix}\}$. This shows nullity of $T$ is $1$.
And also by reducing $A$ to an echelon form (it already is one), the columnspace of $A$ is $$
\text{columnspace}(A) = \operatorname{span} \{\begin{pmatrix}1\\0\end{pmatrix}, \begin{pmatrix}1\\1\end{pmatrix}\}
$$So a basis of $\operatorname{ran} T$ is $\{\begin{pmatrix}1\\0\end{pmatrix},\begin{pmatrix}1\\1\end{pmatrix}\}$. This shows the rank of $T$ is $2$.
We remark that this linear map $T:\mathbb{R}^{3}\to \mathbb{R}^{2}$ has a domain $\mathbb{R}^{3}$ whose dimension is $3$. And $1+2=3$. $\blacklozenge$
This suggests the following theorem.
> **Theorem.** Rank-nullity theorem.
> Let $T:U\to V$ be a linear map, where $U$ and $V$ are linear spaces. Then we have $$
\dim U = \text{rank}(T) + \text{nullity}(T).
$$
Its proof is in the handout given to you on day 1. This is a useful result that relates the dimensionality of three linear spaces.